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1. An image processing device for removing fixed pattern noise in images captured by 
an image pickup device mounted on a vehicle, comprising: 

(a) memory on which is stored a plurality of images captured by the image pickup 
device while the vehicle is running; and 

(b) a controller operativeiy coupled to the memory and adapted to generate 
correction data by extracting high spatial frequency components from portions of the stored 
images, and to use the correction data to remove fixed pattern noise from images captured by 
the image pickup device. 

2. The image processing device of Claim 1, wherein the controller is further adapted to 
use a high pass spatial frequency filter to extract the high spatial frequency components. 

3. The image processing device of Claim l, wherein the controller is further adapted to 
use a low pass spatial frequency filter to extract the high spatial frequency components. 

4. The image processing device of Claim l, further comprising: 

a sensor adapted to be mounted to the vehicle and to output a behavior signal 
indicating whether the vehicle is undergoing a detected type of movement; 

wherein the controller is further adapted to store in memory the images captured by 
the image pickup device in response to the behavior signal indicating that the vehicle is 
undergoing the detected type of movement. 

5. The image processing device of Claim 4, wherein the detected type of movement is a 
yaw movement that is sufficient to blur the images captured by the image pickup device, and 
wherein the controller is further adapted to extract correction data from horizontally-oriented 
portions of the images. 

6. The image processing device of Claim 4, wherein the detected type of movement is a 
pitch movement that is sufficient to blur the images captured by the image pickup device, and 
wherein the controller is further adapted to extract correction data from vertically-oriented 
portions of the images. 
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7. The image processing device of Claim 4, wherein the controller is further adapted to 
store the images in the memory during a time period that varies with the rate of the detected 
type of movement. 

8. The image processing device of Claim 7, wherein the controller is further adapted to 
store the images in memory during a time period that varies inversely with the rate of the 
detected type of movement. 

9. The image processing device of Claim l, further comprising: 

a wiper area extractor adapted to extract, from the images captured by the image 
pickup device, a plurality of areas that correspond to windshield wipers; 

wherein the controller is further adapted to superimpose the plurality of extracted 
windshield wipers areas to form one or more of the stored image portions from which 
correction data is extracted. 

10. The image processing device of Claim 9, wherein the wiper extractor is adapted to 
calculate speed of objects in the images captured by the image pickup device, and to extract 
as the windshield wiper areas the objects in the images having a speed above a prescribed 
value. 

11. The image processing device of Claim 1 wherein the processor is further adapted to 
extract a high spatial frequency component for which the luminance output value is highest in 
the stored images and a high spatial frequency component for which the luminance output 
value is lowest in the stored images, and to adjust the gain and offset of the images captured 
by the image pickup means based on the extracted high spatial frequency component at high 
luminance and the high spatial frequency component at low luminance. 

12. An image processing device for removing fixed pattern noise from images captured 
by an image pickup device mounted on a vehicle, comprising: 

(a) storage means for storing a plurality images captured with the image pickup 

means; 

(b) extracting means for extracting the high spatial frequency component contained 
in the images in the storage means and generating correction data for removing the fixed 
pattern noise based on the high spatial frequency component; and 
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(c) image correcting means for removing fixed pattern noise from the images 
captured with the image pickup means based on the correction data. 

13. The image processing device of Claim 12, wherein the storage means stores images 
captured by the image pickup device while the vehicle is traveling. 

14. The image processing device of Claim 12, further comprising: 

(a) behavior detecting means for detecting the behavior of the vehicle; 

(b) storage time setting means for determining a time period during which images 
captured by the image pickup means will be stored in the storage means based on the 
behavior of the vehicle detected by the behavior detecting means; and 

wherein the storage means stores the images captured by the image pickup means for 
the duration of the storage time set by the storage time setting means. 

15. The image processing device of Claim 12, further comprising: 

wiper area extraction means for extracting the areas in which a wiper is present from 
images captured by the image pickup means; wherein the storage means stores a plurality of 
wiper areas that have been extracted from images by the wiper area extraction means. 

16. The image processing device of Claim 15, wherein the wiper area extraction means 
further comprises: 

speed calculation means for processing the images captured by the image pickup 
means to calculate speed information for the images; wherein the area on the images for 
which speed above a prescribed value is calculated is extracted as the wiper area based on the 
image speed information calculated by the speed calculation means. 

17. The image processing device of Claim 12, further comprising: 

a gain offset means for extracting the high spatial frequency component for which the 
luminance output value is highest in the stored images and a high spatial frequency 
component for which the luminance output value is lowest in the stored images, and for 
correcting the gain and offset of the images captured by the image pickup means based on the 
extracted high spatial frequency component at high luminance and the high spatial frequency 
component at low luminance to improve the uniformity of the images. 
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18. A vehicle having an image processing device for removing fixed pattern noise in 
images, comprising: 

(a) a vehicle having a camera mounted thereto, the camera including an image- 
pickup element generating as output a plurality of images; 

(b) memory on which is stored the plurality of images generated by the image 
pickup element while the vehicle is running; and 

(c) a controller operatively coupled to the memory and adapted to generate 
correction data by extracting high spatial frequency components from portions of the stored 
images, and to use the correction data to remove fixed pattern noise from images captured by 
the image pickup element. 

19. The vehicle of Claim 18, further comprising: 

a sensor mounted to the vehicle and adapted to output a behavior signal indicating 
whether the vehicle is undergoing a detected type of movement; 

wherein the controller is further adapted to store in memory the images captured by 
the image pickup element in response to the behavior signal indicating that the vehicle is 
undergoing the detected type of movement. 

20. The vehicle of Claim 19, wherein the detected type of movement is a yaw movement 
that is sufficient to blur the images generated by the image pickup element, and wherein the 
controller is further adapted to extract high spatial frequency components from horizontally- 
oriented portions of the images. 

21. The vehicle of Claim 19, wherein the detected type of movement is a pitch movement 
that is sufficient to blur the images generated by the image pickup element, and wherein the 
controller is further adapted to extract high spatial frequency components from vertically- 
oriented portions of the images. 

22. The vehicle of Claim 19, wherein the controller is further adapted to store the images 
in the memory during a time period that varies with the rate of the detected type of movement. 

23. The vehicle of Claim 18, further comprising: 

windshield wipers and a wiper controller adapted to generate a wiper activation signal 
when the windshield wipers are operational; 
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wherein the controller is responsive to the wiper activation signal and is further 
adapted to extract areas of the images that correspond to the windshield wipers, and to 
superimpose the extracted windshield wiper areas in the memory to form one or more of the 
stored image portions from which the controller extracts high spatial frequency components. 

24. The vehicle of Claim 23 wherein the controller is further adapted to calculate speed of 
objects in the images generated by , and to extract as the windshield wiper areas the areas on 
the images having a speed above a prescribed value. 

25. The vehicle of Claim 18 wherein the processor is further adapted to extract a high 
spatial frequency component for which the luminance output value is highest in the stored 
images and a high spatial frequency component for which the luminance output value is 
lowest in the stored images, and to correct the gain and offset of the images captured by the 
image pickup means based on the extracted high spatial frequency component at high 
luminance and the high spatial frequency component at low luminance. 

26. An image processing system for removing fixed pattern noise in images, comprising: 

(a) a camera including an image pickup element generating as output a plurality 
of images; 

(b) memory on which is stored the plurality of images generated by the image 
pickup element; and 

(c) a controller operatively coupled to the memory and adapted to generate 
correction data by extracting high spatial frequency components from portions of the stored 
images, and to use the correction data to remove fixed pattern noise from images captured by 
the image pickup element. 

27. The image processing system of Claim 26, further comprising: 

a sensor adapted to output a behavior signal indicating that the camera is undergoing a 
detected type of movement; 

wherein the controller is further adapted to store in memory the images captured by 
the image pickup element in response to the behavior signal indicating the detected type of 
movement. 



-25- 



WO 2006/040662 PCT/IB2005/003056 

28. The image processing system of Claim 26, wherein the detected type of movement is 
a yaw movement that is sufficient to blur the images captured by the image pickup element, 
and wherein the controller is further adapted to extract high spatial frequency components 
from horizontally-oriented portions of the images. 

29. The image processing system of Claim 26, wherein the detected type of movement is 
a pitch movement that is sufficient to blur the images captured by the image pickup element, 
and wherein the controller is further adapted to extract high spatial frequency components 
from vertically-oriented portions of the images. 

30. The image processing system of Claim 27, wherein the behavior signal indicates the 
rate of the detected type of movement, and the controller is further adapted to store the 
images in the memory during a time period that varies with the rate of the detected type of 
movement. 

3 1. The image processing system of Claim 26, wherein the controller is further adapted to 
extract areas of the images that correspond to windshield wipers, and to superimpose the 
extracted windshield wiper areas in the memory to form at least one of stored image portions 
from which the controller extracts high spatial frequency components. 

32. The image processing system of Claim 31, wherein the controller is further adapted to 
calculate speed of objects in the images generated by the image pickup element, and to 
extract as the windshield wiper areas the areas on the images having a speed above a 
prescribed value. 

33. The image processing system of Claim 26, wherein the processor is further adapted to 
extract a high spatial frequency component for which the luminance output value is highest in 
the stored images and a high spatial frequency component for which the luminance output 
value is lowest in the stored images, and to correct the gain and offset of the images captured 
by the image pickup means based on the extracted high spatial frequency component at high 
luminance and the high spatial frequency component at low luminance. 
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34. A image processing method for removing fixed pattern noise from images captured 
by an image pickup device mounted on a vehicle, comprising: 

(a) storing a plurality of images captured with the image pickup means; 

(b) extracting the high spatial frequency component contained in the images; 

(c) generating correction data for removing the fixed pattern noise based on the 
high" spatial frequency component; and 

(d) using the correction data to remove fixed pattern noise from the images 
captured by the image pickup. 

35. The image processing method of claim 34, wherein storing the plurality of images 
takes place while the vehicle is moving. 

36. The image processing^ method of claim 34, further comprising: detecting the behavior 
of the vehicle; based on the behavior of the vehicle, determining a time period during which 
images captured by the image pickup means will be stored; and storing the plurality images 
during the time period. 

37. The image processing method of claim 34, further comprising: extracting the area in 
which a wiper is present from images captured by the image pickup device; superimposing 
the wiper areas to define an image having at least one portion that is black; and extracting 
high spatial frequency component contained in the black portion. 

38. The image processing method of claim 34, further comprising: processing the images 
captured by the image pickup means to calculate speed information for the images; and 
extracting as wiper areas the portions of the images for which speed above a prescribed value 
is calculated. 

39. The image processing method of Claim 34, further comprising: extracting a high 
spatial frequency component for which the luminance output value is highest in the stored 
images and a high spatial frequency component for which the luminance output value is 
lowest in the stored images, and improving the uniformity of the images by correcting the 
gain and offset of the images based on the extracted high spatial frequency component at high 
luminance and the high spatial frequency component at low luminance. 
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